PrismML unveils Bonsai 27B, a compressed AI model built to run on a phone

PrismML has released Bonsai 27B, describing it as the first 27B-class AI model small enough to run within the memory limits of a consumer smartphone. The binary build comes in at 3.9 GB and runs at 11 tokens per second on an iPhone 17 Pro Max, while a 5.9 GB ternary version reaches about 26 tokens per second on an M5 Pro laptop. Both are available for free under Apache 2.0. The company said the model relies on compression technology based on Caltech intellectual property, reducing model weights from 16-bit floating-point precision to either binary or ternary values while preserving most of the original model’s output quality. PrismML also said Bonsai compresses embeddings, attention, and the language model head end to end, unlike many low-bit systems that keep sensitive layers at higher precision. In benchmark tests across 15 tasks on NVIDIA H100 GPUs, Ternary Bonsai 27B scored 80.49, or 94.6% of the full-precision model, while the 1-bit version scored 76.11. PrismML CEO Babak Hassibi told CNBC the company is in early talks with Apple, which is evaluating the compression technology for possible on-device use. Hassibi also said a compressed Gemma model is next, followed by larger frontier models.